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SOME RECENT IDEAS IN RESEARCH METHODOLOGY 

( < paper for a conference on "New Directions in Research 
cn Educational Administration" sponsored by CASEA and UCEA 
at the University of Oregon, March 29-31, 1965) 

Philip J. Runkel 

The purpose of this paper is to describe very briefly a couple of 
particular ways of thinking about research methodology. My selection of 
these ways of thinking is purely arbitrary, but I have chosen these par- 
ticular ideas because first, I consider them very powerful ideas and sec- 
ond, they are not yet being widely taught and are, therefore, not easy to 
come by. Not only do these ideas help those familiar with them to seek 
connections and make comparisons among the plethora of methods being used 
in current research, but in my opinion these ideas will come more and 
more to be used as tools by those who will be developing new methods to 
replace those with iidiich we have become dissatisfied. 

Each of the two schemes of thoug^it idiich I shall present is very 
simple in its basic structure, though very extensive and complicated in 
its implications. The first idea is that of facet design and analysis 
originated by Louis Guttman, now director of the Israel Institute of 
Applied Social Research. Guttman is now in the process of completing 
the first draft of a book about facet design and analysis, but until that 
book appears the only published infonnation about this idea is to be found 
in a few scattered articles. The second idea is the theory of data of 
Clyde H. Cocmobs of the Psychology Department at the University of Michigan. 
Most of the structure so far worked out for tnis theory of data is now con- 
tained in a book ^ich appeared Just last year. Obviously, only the brief- 
est of introductions to each of theso ideas can be contained in this paper. 
References idiich can be pursued if either of the ideas strike you as useful 



will be fo\md at the end of the paper. 

It may be that you are already familiar vrith these ideas. If so, 
we shall be grateful for youi* help during the discussion sessions; the views 
of more than one person famililar vdth an idea will enrich the discussion. If 
all these ideas are new to you, they will probably compound into an overdose; 
if this is the case, I suggest that you concentrate your thinking upon one 
idea vAiich most captures your interest. 

FACET DESIGN AND ANALYSIS 

Facet deoii!n is a way of laying out a domain for research. Although 
it produces hypotheses, its special power as a tool resides in the fact that 
it enables one to specify the boundaries and structure of the entire domain 
of relevance within \diich one may wish to experiment. Facet analysis then 
enables one to test the validity of one's assessment of the entire domain 
before he invests great time and money in experiments upon por*tions of the 
domain. But I have fallen into rather abstract phrases. In order to turn 
to a more concrete way of speaking, let me take a couple of paragraphs to 
set the stage with a few familiar features of experimental design. 

In designing an experiment, one of the most nagging problems is 
Aether we have chosen (a) relevant variables to be allowed to vary and 
(b) relevant variables to be controlled. That is, our hypotheses typi- 
cally take this form: under what conditions will values of a certain 

(dependent) variable be hi|^er and under what conditions will the values 
be lower? For example, we mi^t ask under udmt conditions teachers will 
stay longer in a school and under iidiat conditions teachers will cut short 
their stay. Me mli^t feel that the amount of conmunication of seme cer- 
tain kind within the faculty would be related to length of service — 
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and we mi ght. j therefore, produce or look for schools in idiich there were 
conditions of low, medium, and hi^ communication • For purposes of econ~ 
remy in research, we might want to rule out the effects of other conditions 
idiich we felt would also have an effect on length of service. We mij^t 
believe that salary levels would be related to length of service but, if 
we were primarily interested in the effects of communication, we would 
either randomize salary levels among our subjects or we would examine 
length of service within groups of teachers having salary levels relativ- 
ely the same. The point is that we would pay attention in one way or 
another both to communication levels and salarjr levels (as well as other 
variables we think might be relevant) and the conclusions we would draw 
from our study would necessaid-ly be circumscribed by these relevant vari- 
ables. If we hold a variad)le constant at one level, then our conclusions 
must speak of our findings within this particular slice of total domain. 

If we select subjects landomly from among a certain set of conditions, then 
our conclusions are complete only if they state that we have ignored the 
effects of this variable (which we think might have had effects had they 
been allowed to show). To repeat, we begin a study by trying to become 
aware of the variables vdiich are i«5)ortant in circxanscribing the domain 
^ich we want to study. 

It is not easy to become aware of tho variables idiich put the 
“boundaries” on the domain we want to study. One of the difficulties is 
the ever-present bias on the part of the experimenter toward his own par- 
ticular ways of looking at the world. Selecting variables (a) to be ex- 
amined, (b) to be randomized, and (c) to be ignored typically becomes a 
matter of intuition. Another difficulty arises from the very technique of 
randomization in assigning subjects to experimental conditions. Randomizing 
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rules out from, observation the effects not only of variables »hich we are 
aware mi^t be relevant but also the effects of those of which we are un- 
aware. \ilhat we need is some method vAiich will enable us to be systematic 
in thinking about the domain vdiich we are setting out to study . 

The beginning logic of facet design is extremely simple; it is 
singly that of the Cartesian coordinates you used when you drew graphs 
in algebra class in Junior high school. The chief differences are (a) 
that we do not always deal in the social sciences with numerical quantities 
along the coordinates and (b) we typically deal with many more coordinates 
than we were able to put on the graph in junior hi£^ school. 

To be more exacts we mi^t list down the left-hand side of a page 
some conditions of one sort and across tho bottom of the page some con- 
ditions of another sort. By drawing horizontal and vertical lines, we 
would then have a grid of cells. Each cell would be designated 1::^ a par- 
ticular conoition of the one sort and a particular condition of the other 
sort. For example, we mi^t list levels of communication down the left 
side of the page and levels of salary across the bottom of the page; in 
a particular cell we would then enter observations of length of service 
on the part of those teachers \dio were characterized by that particular 
salazy level and that particular amount of communi cation in his school. 
Comparisons of length of service in relation to level of communication 
with salary held constant could then be made up and down a column of the 
diagram, while comparisons of length of service in relation to salary 
levels with level of communication held constant could be made from left 
to ri^t along the rows of the diagram. Comparisons taking into account 
both commini cation and salary would be made cLtagonally. In this example, 
comauni cation is one facet and the various levels of communication ai‘e the 
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elements of that facet. Salary is the other facet. The variable of length 



of service is, of course, the dependent variable and the values of this 
variable constitute what Guttman calls the "range.” 

Taking this example a little farther, let us suppose for the sake 
of illustration that we predict that length of service will be longer where 
levels of communication are hi^er and also where salaries are high^z** W© 
iiould then expect that the average length of service would increase in the 
cells reading froia left to right along a row and would also increase in 
the cells running from bottom to top in a column. Furthermore, we would 
predict that the average length of service in any particular cell would 
be higher ^han in cells to the "southwest" of it. On the other hand, we 
would not be able to predict the relative magnitudes to be found in cells 
to the northwest or southeast because the elements corresponding to those 
cells would be lower in one facet but hi^er in the other and therefore 
incomparabXe Facet design does not lay out all possible predictions 
about all sets of conditions specified by the facets. 

There is another important feature to be observed in this example. 
Since the values for length of service increase along a row, the values 
in one cell are more similar to the values in an adjacent cell than they 
are to values in cells farther avay. The same kind of pattern will be true 
of columns and it will also be true diagonally across the diagram. This 
pattern reflects vdiat Guttman calls the principle of contiguity . vdiich is 
the basis for facet analysis . The principle is usually expressed in terms 
of correlations between observations in different cells rather than in 
terns of levels of values. 

By now I have presented enou^ ideas about facet design and analy- 
sis so that it will probably be profitable to present a more extensive 




example. Quoted below (with minor changes in wording) are excerpts from 
an article by Guttman recently published in the American Sociological Re- 
view . This article takes data previously published by other authors, re- 
casts them into a facet design, and subjects them to a facet analysis. 

The original article written by the other authors reported a study con- 
ducted vrithin a domain of interest conceived in the usual intuitive man- 
ner. Guttman 's stuc^y of their study shows clearly how the domain can be 
specified much more precisely, how a family of hypotheses flows directly 
from the facet desigri, and how the validity of the facets originally chosen 
can be checked in one coherent facet analysis. You should pay particular 
attention as you read along to the manner in viiich the facet design illumi- 
nates the domain of interest well beyond that portion of the domain which 
was investigated by the original authors. 

I might make a note here about the process of choosing facets 
because, sometimes, facet analysis is confused with factor analysis. 

In the latter, factors are come upon after the data are collected. Quite 
the contrary is true of facet design. The facets must be chosen before 
data are collected. This is done, to put it brutally, as best one can. 
However, there are aids upon which one can rely. One aid, of course, is 
a theory about the domain being studied — if one is available or can be 
built. Another way to go about it, which is exemplified in the article 
excerpted below, is to study the materials available concerning the do- 
main, looking for concepts vdiich, viien put together in the criss-crossing 
manner of the Cartesian space, will specify all the sets of conditions 
vdiich one has foixnd in his inspection of the domain. Note in the follow- 
ing article that the original authors did not provide Guttman with the 
facets vdiich he used in his re-analysis. Rather, Guttman inspected the 
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conditionsi described by the original authors in their article and the kinds 

\ 

of data-gathering instruments they used. He then picked o\xt the features 
(facets) vdiich he felt would "span" the conditions under which the original 
authors were studying favorableness of behavior or attitude toward Negroes. 
When this was done, it was immediately apparent that the facets spanned 
more combinations of conditions than the original authors had studied. In 
other words, the facet design laid out the complete domain of which the 
conditions studied by the original authors constituted merely a part. But 
perhaps I am talking too much about a study vAiich you have not yet read a 
Let us turn to the excerpts from Guttman. 
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In a recent article, Bastide and van den Berghe describe 
four types, or sub\iniverses, of content in connection with inter- 
racial behavior: stereotypes , norms , hypothetical interaction , 

and personal inter action * They present some interesting empiri- 
cal findings in these areas, based on their Brazilian research, 
including a set of correlations among the four subuniverses. 

The purpose of the present paper is to suggest a structural 
theory for the observed interrelations among the four subuniverses. 

The four varieties of interracial behavior on which we are 
focussing are described briefly by Bastide and van den Ber^e. 

For our present purposes, it is convenient to restate and to 
capsulize the descriptions. The following are definitions of the 
subuniverses of the four types: 

I. Stereotype: Belief of (a white subject/ that his own 

group (excels — does not excel) in comparison with 
Negroes on (desirable traits) . 

II. Norm: Belief of (a white subject) that his own group 

(ou^t — ought not) interact with Negroes in (social 

ways) . ^ . 

III. Hypothetical Interaction: Belief of (a white subject) 

that he himself (will — will not) interact with Negi’oes 
in (social ways). 

IV. Personal Interaction: Overt action of (a white subject) 

himself (to — not to) interact with Negroes in (social 
ways) . 

Underlinings and parentheses used in the above definitions 
are intended to indicate the semantic structure we are positing 
for intergroup behavior, and each will be explained. 

All four definitions have in ccmmon the fact that they in- 
volve a white subject and Negroes. The phrase "with Negroes" 
occurs unifomly in each of the definitions, as does "a white sub- 
ject." The intergroup behaviors of some pairs of groups other 
than whites and Negroes can be defined merely by replacing "white" 
and "Negroes" with the respective characterizations of the de- 
sired groups. It is convenient to think of specific groups, such 
as Negroes and whites, in developing our theory. The theory may 
be enlarged subsequently by letting the groups vary according to 
some principle . 

The common or fixed elements indicate the universe of which 
the subuniverses are subsets. Since the subuniverses nevertheless 
differ among themselves, our first task is to ascertain the facets 
which deteimine those differences. 

The four definitions differ among themselves primarily on 
three facets, the elements of ^ich are underlined in the defini- 
tions glvm above . Each definition concerns a type of behavior 
of a subject vis-a-vis a type of intergroup behavior of a type of 
referent. Two kinds of behavior for the subject occur in the 
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definitions: belief (a form of covert behavior) and overt action . 

Also, two kinds of referents occur: the subject's group and the 

subject himself , Simi-Larly, two kinds of intergroup behavior are 
distinguished: comparative and interactive , 

Thus, each of the three facets occurs as a dichotomy. It is 
helpful to list them and their elements in tabular form, assi@aing 
symbols to each for later use as in Table 1, 



TABLE 1, Facets on Which Subuniverses Differ 



A ~ Subject ' s Behavior B - Referent C = Referent ' s Intergroup Behavior 



ai == belief bj^ = subject's group c^ = comparative 

^2 ~ overt action ~ subject himself C 2 = interactive 



The capital letters A, B, and C denote the three facets, \AiXle the 
corresponding small letters with subscripts denote the elements of 
the respective facets. 

Three dichotomous facets yield eight (=2x2x2) possible 
combinations of three elements each, one element from each facet. 
That is, the Cartesian product of the three facets, which may be 
denoted by ABC, is a set of eight profiles, each profile having 
three components. Each profile defines a different subuniverse. 
But we have defined above only four subuniverses, or only a sub- 
set of ABC, We ma 3 »- tabulate this subset explicitly as follows: 





Subuniverse 


Profile 


I, 


Stereotype 




II, 


Norm 




III, 


Hypothetical Interaction 




17, 


Personal Interaction 


“2”2®2 



A striking feature revealed by this tabulation is that the 
profiles fom a perfect scale. For each facet, a single cutting 
point in the rank order of the subuniverses suffices to divide 
the elements with subscript "1" from the elements with subscript 
"2," For facet A, the cutting point is between III and 17; for 
facet B, between II and III; and for facet C, between I and II, 
If we arbitrarily call subuniverse I the left end of the scale, 
we have the rank order I <II < III <17, or more verbally: 
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stereotype < Norm < Hypothetical Interaction C Personal Interaction. 
The symbol "<" is used merely to signify that what is to the left 
of the symbol precedes what is to the right of the symbol in the 
rank order. 

Scale structures have been found in various kinds of empirical 
data — in ranking people, social institutions, political units, 
and so on. We now find the same structure in ranking abstract con- 
cepts. In the empirical cases, only approximately perfect or 
quasi-scales are the rule, which sometimes raises troublesome pro- 
blems of how to handle scale error. On the nonempirical, concep- 
tual level, truly perfect scales are not at all impossible, as 
exemplified above, with no problem of scale error. 

Semantic meaning for the rank order requires exploration. 
According to scale theory, ordering the profiles also implies a 
formal ordering of the categories within each facet. The ordering 
I < II <III < IV implies formally the three simultaneous orderings: 
^1 < ^ 2 * ^1 < ^2^ ®1 these further orderings have any 

common semantic meaning, formalisms aside? If not, the formal 
scale of the four subuniverses need have no clear semantic meaning. 

A common meaning for the orderings can be suggested: they 

show in each case a progression from a weak to a strong form of 
behavior of the subject vis-a-vis Negroes. "Belijf" is weaker 
than "overt action" in being passive rather than active. Referring 
to the behavior of "subject's group" is weaker than the subject 
referring to "himself," insofar as the subject's relations with 
Negroes are concerned. "Comparative" behavior is weaker than 
"interactive" behavior since it does not imply social contact; a 
comparison is more passive than interaction. 

Accepting this interpretation of the orderings within facets, 
we can say that the ordering of the subuniverses themselves also 
rxms from weakest to strongest. "Stereotype" is the weakest 
form of intergroup subuniverse, "Personal Interaction" is the 
strongest form, while the other two subuniverses are intermediate 
in strength, in the indicated c-rder. 

What of the profiles from ABC that were not included among the 
original four definitions? These can define further subuniverses. 
Thus : 



Two of the omitted profiles are a2^1®l ®^2^1®2* Each of 

these, according to its first two components (a 2 b 3 ^), concerns overt 
action (a 2 ) of the subject with reference to the subject’s group 
(bi) . Presumably, each implies that the subject should activate or 
stimulate his group — in one case as to how it compares itself with 
Negroes (cj^), and in the other, regarding interaction with Negroes 
(c2). The first case might be called "Teaching" and the second 
"Preaching. " 

The remaining two omitted profiles are aQ^b 2 Ci and a 2 b 2 CQ^. 
According to their last two components (b 2 CQ^), each of tnese'^con- 
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corns the comparison (ci|^) of the subject himself (b 2 ) with Negroes 
as to desirable traits. The first profile might be called “Feel 
Superior" and the second "Act Superior." 

Six scales are possible, given the orderings < a 2 , b^ < b 2 . 
Cl <*C 2 for the three facets. One way of bringing this out is to 
portray the entire universe as a partly ordered set, as in Figure 1. 



Figure 1 



Level I: 



Level II: 



Level III: 



Level IV: 



Hypothetical 

Interaction 

aib2C2 



Norm 

aibiC2 




Stereotype 



Teaching 

aabici 



Feel Superior 
^ 1 ^ 2°1 



Preaching 

aobn Co 




Act 
SiTperior 




Hypothetical 

Interaction 

^1?2°2 



Personal Interaction 



^2 °2 



The connecting lines between "levels" in the figure show the pairs 
of profiles between which the inequality relation "<" holds. If 
abc and a'b'c' are any two different profiles, then abc ca'b*c' 
if and only if simultaneously a^ ' , bS^ ' , and c^c ' . 

In a scalable series of levels, each profile differs from its 
predecessor on only one facet. Starting with Stereotype as Level 
I, this allows for only three possible profiles for Level II, for 
there are only three subscripts in Stereotype that can be raised 
from a "1" to a "2." Since each profile in Level II has only two 
subscripts of "1" that can be changed to a "2," there are only two 
ways that a profile can move to Level III, so each has tifo lines 
branching down from it. For convenience, we have listed Hypothe- 
tical Interaction twice on Level III in the figure in order to 
avoid criss-crossing lines. There are only three profiles possible 
at Level III, since there are only three ways of having one sub- 
script equal to "1" and the remaining two equal to "2." The scales 
are the six continually descending possible pathways from Stereo- 
type to Personal Interaction in the diagram, along the lines run- 
ning from level to level. 

The universe is only partly ordered, for many pairs of pro- 
files are not comparable. For example, we cannot say that the 
inequality "<" holds between Norm and Act Superior, in either 
direction; while the latter exceeds the former on facets A and B, 
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the reverse is true on facet C. Any two profiles of the same level 
or noncomparable for sdmilar reasons. 

Bastide and van den Berghe apparently carried out no tests of 
scalability for their data. It seems plausible, however, that each 
of their subuniverses should be at least quasi-scalable . For pur- 
poses of our present discussion, let us accept the hypothesis that 
numerical scores may be sufficient for studying the empirical inter- 
relationships among the subuniverses. Let as accept the empirical 
correlation coefficients of the Brazilian study as a correct pic- 
ture of the relationships among the subuniverses consiaered. 

The empirical product-moment correlations between the eight 
subuniverses of ABC for population P may be calculated. Our 
analysis above of the semantic structiire of ABC provides a social- 
theoretical basis for predicting, the structure of this empirical 
correlation matrix. One cannot presume to predict the exact size 
of each correlation coefficient from knowledge only of the semantics 
of universe ABC, but we do propose to predict a pattern or struc- 
ture for the relative sizes of the statistical coefficients from 
purely semantic considerations. 

Since the Brazilian data allow us to check our prediction 
only with respect to the int ere orrelat ions of scores on four of 
the subuniverses (those earlier labeled I, II, III, and IV), we 
concentrate first on these. Our prediction is based on the fol- 
lowing proposition: 

Contiguity Hypothesis . Subuniverses closer to each other in 
the semantic scale of their definitions will also be closer 
statistically . 

Statistical closeness is measured approximately (but not 
always exactly, as we shall iidicate below) by correlation co- 
efficients. According to the contiguity hypothesis, generally 
Stereotype should correlate more highly with Mora than with 
Hypothetical Interaction, and almost certainly more than with 
Personal Interaction; rj jj > rj in>rj Similarly, we 



conti- 
guity are defined by semantic rank order considerations for the 
subuniverses . 

The actual correlation matrix reported by the authors of the 
Brazilian study is equivalent to the presentation in Table 2. 

The structure of this correlation table is virtually as predicted. 
The largest correlations tend to be adjacent to the main diagonal, 
corresponding to the semantically contiguous subimiverses — and 
they taper off to the northeast and southwest comers of the 
table, where semantic differencee increase. 
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TABLE 2* Etoplrlcal IntercorrelatlonB of Scorea 
on the Four Subimlveraea 





I 


II 


III 


IV 








Hypothetical 


Personal 


Subuniverse 


Stereotype 


Norm 


Interaction 


Interaction 


I. Stereotype 


- 


.66 


37 


.25 


II. Norm 


.60 


- 


.68 


.51 


III. Hypothetical Interaction 


.37 


.68 


MW 


.49 


IV. Personal Interactioii 


.25 


.51 


.49 


— 



An apparent sli^t exception to this structure is that r^ jjj 
(= .49) does not quite exceed vjjj .5l)> despite the fact 

that semantically III lies between II and IV. This need be no 
actual contradiction of the contiguity hypothesis, as should be- 
come evident in the explicit discussion of statistical distances 
below. Sampling error, of course, might also be offered as a 
tentative explanation for the apparent aberration, but we prefer 
to ignore such "outs" for the purpose of the exposition of our 
theory. The selection of the 580 subjects used fl’om the Brazilian 
population is described as "neither random nor proportional;" even 
this need not affect our structural theory. The relative pattern- 
ing of correlations need not change even though the absolute sizes 
of the coefficients might. Idiosyncracies of sampling and biased 
selections of subjects often can violently affect arithmetic means 
and other averages, as well as variances and other measures of 
dispersion. Correlation coefficients are often harder to destroy 
or to build up artificially, and the possible attenuation of dis- 
attenuation is even less likely to alter the pattern of correlations, 
since such effects usually influence correlation coefficients by 
constants of proportionality. 

» 

Had we empirical data for the complete matrix of correlations 
among all eight subuniverses, the validity of our structural theory 
as thus far developed could be checked further. The empirical 
results could be compared with the calculated results, and thus 
verify the adequacy of the structural theory. 

Failure of the results to check out could imply either or 
both of the following possibilities: the statistical structure 

deduced from the semantic structure was not appropriate; the 
semantic structure was faulty or incomplete. 

Note that the facet design, in contrast to the original limited de- 
sign, lays out all the sets of conditions (or "subuniverses") vrtiich are 
relevant to the kind of hj-pothesis chosen by the original authors. The facet 
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deeign immediately produces an entire family of hypotheses; namely^ those com- 
parixig "favorableness" in any two or more of the ei^t cells or subuniverses. 
The use of the principle of contiguity in the analysis makes an immediate 
check on Aether the orclering of the elements in the several facets were pro- 
perly chosen^ and ^diether the facets and their elements were valid in the 
sense of supporting the pi^ediction which the principle of contiguity makes . 

The principle of contiguity, in turn, is merely a direct inference from the 
meaning of "ordering" the elements within the facets. 

The facet' design in the study quoted above clearly exemplifies 
another advantage of this tool. In the restatement by Guttman of the de- 
scriptions of the subuniverses (labeled in Roman mmierals in the second 
paragraph of the excerpts) it is easy to see some kinds of facets which could 
be added so as to enlarge the domain of study. In fact, each phrase enclosed 
in parentheses in the descriptions of the subuniverses suggests another kind 
of facet which mi^t be explored. The particular elements ^ich mi^t be 
chosen to compose these facets are, of course, up to the ingenuity and 
theoretical acumen of the investigator. 

Let us now turn to another application of facet design. Some time 
ago Guttman 's institute was given the task by the Israeli government of 
developing guides or rating sheets which could be used to establish the 
salary levels of jobs within the Israel Civil Service, After a great deal 
of interviewing and studying job specifications to detemine what people felt 
was meritorious about the duties and skills demanded by the various jobs, 
Guttman and his colleagues decided upon the five facets labeled with capital 
letters on the next page. The next task was to develop a set of items based 
on these facets which coiild be used to evaluate the various jobs in the system. 
Once the facets and their elements were chosen, the construction of items for 
evaluating job level was strai^tforward. 
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FACETS F(B JOB EVALUATION 
(by L. Guttanan for Israel Civil Service) 



Types of restriction on work (facets): 



B 



Fj^eedoft 
Freedcift to 
chan#? thiPAis 



Guideline 



Subordination 
to supervisor 



Supezvision 



ai transDlt 
as is 



i>£ '<tetaiied "" c^^ receives 
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The facets described in the tabulation specify 3x4x3x3x3 or 
cells; that is^ 324 types of jobs. Since the elements within each facet 
are ordered according to level of restriction on work, cells specified by 
elements with larger subscripts are at higher levels than cells specified 
by elements with smaller subscripts. Accordingly, an instrument or rating 
sheet can be devised on which each item carries the specifications of some 
cell out of the 324. However, not all of the 324 possibilities would be use- 
ful; as we have seen earlier, not every cell in the complete design can be 
said to be higher or lower than every other. Furthermore, one encounters 
practical difficulties which reduce the number of items which might be used, 
such as diffidulties in phrasing items, amount of time or fatigue required 
to answer the items, and so forth. 

The nature of the items finally selected is shown in the lower half 
of the tabulation. Each combination of small letters characterizes an item. 
For example, a^^bj^ would sound something like, "A person in this job passes 
on information or materials without change according to detailed instructions 
provided him." The item a^b^ might read simply, "Creates policy." The item 
b^c^d^ might read, "Periodically, the person in this ^’ob seeks advice from 
others of his own choosing on whether his actions are carrying out the 
policy directives under which he works." Each line across the page in the 
lower half of the tabulation contains items which are at the same "level;" 
items lower on the page represent jobs of higher level than items hi^er on 
the page. All told, the lower half of the tabulation contains 24 items dis- 
tributed over twelve levels. 

Again, the principle of contiguity can be used to check the validity 
of this design. The principle of contiguity states that jobs described by 

4 

adjacent cells of the facet design should be close together in their salary 
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ranges. If it turns out in practice to be difficult to maintain this rela- 
tion among salary ranges, then the evaluation scheme will have been shown to 
be invalid. 

The example just given shows how a facet design can be used to generate 
measuring instruments and also shows the fact that the facets the investiga- 
tor uses for thinking hie way through the problem do not restrict the manner 
in which the investigator need talk to the people he is studying. The 
types of items in the lower half of the tabulation are headed with titles 
which are in terns more familiar to personnel workers than are the titles 
of the facets. 

Let us now turn to another example. In a study concerning communi- 
cation in high school faculties which I am at present pursuing, one of the 
hypotheses is as follows: 

Hypothesis 1. Given pairs of persons in diagreement at Time 1, 
such pairs which remain in disagreesment will be found at Tame 2 
not to have increased their communication, while pairs which in- 
creased their agreement will be found not to have decreased 
their communication. 

This hypothesis is embedded in a domain which (because of seme 
other hypotheses) also includes factors of the social support an individual 
receives for his opinion from other persons. The facets for this domain 
are shown on the next page . Instead of merely listing the facets with their 
elements, a form different from the earlier examples is used in this diagram. 
The elements of the facets (labeled with capital letters) are embedded in a 
sentence which describes the kind of doioain which is being investigated. 

The capital letters A throu^ G designate the facets; the letter R does not 
properly stand for a facet but rather for the "range" or dependent variable. 

It will be seen in the beginning of the sentence composing the diagram that 
certain possible facets are being held constant; that is, this is really a 
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BEHAVIORAL DOMAIN FOR HYPOTHESIS 1 ^ 

Given that respondent^ is an individual in strong disgx*eement with res]pondent2 
at Time 1 in respect to the appropriate duties of the counselor, then; 

A B C 

respondent-j ^ (who is a teacher receiving stronig social support 

respondento counselor moderate 

administrator weak 




D 

another, where the other is receiving strong social support and is a 

moderate 

weak 



E 

teacher ) 
counselor ) 
administrator ) 

R G 

of its frequency and in respect to general commmication . 

teaching-topics conamunication . 
guidance ccmmunication . 



F 

reports his communication with the other at Time 1 in terms 

the other at Time 2 
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subdcjmain of a conceivable larger domain. 

A great variety of sentences can be made from the sentence in the 
diagram simply by picking, under each capital letter, jurt one of the elements 
listed. An entire family of hj^otheses is thereby generated simply by perform 
ing the comparison called for by any two different sentences in respect to 
frequency of communication. It is clear that the diagram shows a great 
many questions which would remain unanswered if the investigator were to 
limit his attention to the hypothesis as originally stated. 

Now let us turn to a final example. Another job which was put into 
the hands of the Israel Institute of Applied Social Research was to develop 
an interviewing procedure which would select teachers who were applying for 
training to become vocational guidance counselors. The following pages 
describe the procedure finally worked out . 

Now I am going to ask you to do something with these ideas. 

EXERCISE 1. Study the pages describing the interview procedure for 
selecting teachers to become counselors. Then make a list of the facets, 
along with their elements, upon which you believe thils procedure to be based. 

It will be helpful if you can type your list upon a ditto master and 
send it to us before the conference; we can then have copies made and dis- 
tributed to the other members at the conference. If you read this ][)aper^ 
too late to send your list on ahead, please bring it vdth you Monday morning 
and we shall immediately have copies made . You may, if you wish, omit your 
name from the shebt you give us. It wi3JL be interesting, I am sure, for 
participants in the conference to see what facets other people saw in this 
interviewing procedure; it is not necessary, however, that they know who 
saw what. 

If you read this paper too late even to think about making ai list, 
don’t worry about it; doing your homework before the conference is not a 
prerequisite to admission. Furthemore, we realize that we are not giving 
you very much time to get it done. I hope, however, that you will find 
time to give this exercise a try, because you cannot very well get the 
"feel" of facet design without trying it yourself. 




The Government of Israel 
The National Institute for Vocationiil Guidance 

Interview of Counseling Teachers 
Instructions for the Intervi<sw 



a) Systematic interchange of tasks among the metnbers of the acceptance 

committee * 

1) Eveiy member of the committee will serve in one of the following 
tasks in each interview; (a) interviewer, (b) rater with form 
'•A," and (c) rater with form "B.” 

2) The tasks of each interviewer will be interchanged at the end of 
each interview. The committee member who was an interviewer at 

a certain interview will serve as a rater at the following inter- 
views, until his turn comes again to serve as an interviewer. 

3) A member ^o was a rater with form "A" at a certain interview 
will rate the next interview with form "B" and vice versa (until 
his turn comes again to be an interviewer). 

The k forms of the interview . 

1) Attached herewith are 4 parallel forms of an interview each of 
which includes 9 questions. The first interviewer employe form 
No. 1, the second employs form No. 2, etc. The fifth returns to 
form No. 1 and so on. 

2) Before the beginning of each interview, the rater must mark on 
his rating fom, at the proper place, which form of interview was 
used. 

3) Before the beginning of the interview, each interviewer has to 
take care that the rater marks properly on his form the .type of 
interview that is being used. 

c) Filling out the rating forms . 

1) At each interview, half of the raters will fill out rating form 
"A," and the other half will fill out form "B." If the number of 
raters is not even, it is necessary to arrange that the fom will 
be filled out evenly in the course of the whole set of interviews. 

2) Before each interview it is necessary to take care of the inter- 
change of tasks among interviewers and raters and also among raters 
in accordance with the instructions of paragraph "a" above. 
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Interview of Couneeling Teachers 
Form 1 



1) What are the things in your present work that you are satisfied with, 
and why? 

2) What do you see as the main function of ths teacher? 

3) Describe the relations that have developed between you and the children 
with idiom you worked last year. 

4) Tell me about your work with children in general* 

5) In vdiich things was your work with pupils successful and in idiich were 
you less successful? 

6) To what extent do you have, according to your opinion, the necessary 
requirements for a good teacher? 

7) Describe your past achievements in teaddng. 

8) To vdiat extent did your class achieve the^ curriculum requirements? 

9) Tell about a child vftiose specific personal problem you have treated. 
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Intervitif of Counseling Teachers 
Form 2 



1) Tell about the problems you have encountered in your work. 

2) Vnmt are the problems that you may encounter in counseling work^ 
according to your opinion? 

3) Did you have an occasion to know children yitio needed counseling espe- 
cially? Describe your relations with them. 

4) Tell about ycur work with children in general. 

5) Did you have an occasion to know children idio need counseling? Can 
you describe their problems? 

6) To \diat extent, do you think, you have the necessary traits of a good 
counselor? 

7) Describe the teaching achievements you had in the past. 

8) On vrtiat did it depend that your class did not achieve in the curric- 
ulum more than it did? 

9) Concerning personality, what differences did you find between the 
children that you taught and rural children, urban children, and 
children from other institutions? 
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* Interview of Counaeling Teachers 

Fom 3 



1) What are the things in your present work that you are not satisfied 
with, and why? 

2) What are, in your opinion, the characteristics of a good teacher? 

3) Describe by what method you succeed in gaining the confidence of the 
children with ^om you work. 

4) Tell about your work with children in general. 

5) In which things did you succeed in your work with pupils, and in 
idiich things have you been less successful? 

6) What is it that gives you satisfaction in your work as a teacher? 

7) Describe your past achievements as a teacher. 

8) To what extent did your class achieve the curriculum material? 

9) Tell about a child >diose specific personal problem you have treated. 
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Interview o£ Cpuneeling Teachere 
Form 4 



1) tell about your work. 

2) VQiat are^ in your opinion^ the characteristics of a good counselor? 

3) Descilbe the relations that have developed between you and the child- 
ren with vAiom you worked last year. 

4) Tell me about your work viith children In general. 

5) Bid you ever have an occasion to get acquainted with children ^o 
needed counseling? Can you describe their problems? 

6) What do you find especially Interesting in your work as a teacher? 

7) Describe your previous achievements in teaching. 

8) l^on vdiat did it depend that yoiu* class did not achieve some of the 
curriculum matezlal? 

9) Concerning personality, what differences did you find between the 
children that you taught and iniral children, urban children, and 
children from other institutions? 




Rating Form of the Interview 
Type A 



Interviewee 




Name of Rater 


Place 




Date 


Form of Interview 12 3 


4 


No. of auestion 


* 




1234567 8 9 


a. Facts only 


1 




Mainly facts 


2 




Many opinions 


2 




Opinions onlir 


4 




b. Achievements only 


1 




Mainly achievements 


2 




Achievements and problems 


2 




Mainly problems 


4 




Pi?oblems only 


§ 




Neither of these 






c. Almost all about himself 


1 


- 


A lot about himself 


2 




Little about himself 


2 




Almost nothing about 
himself 


4 




d. Almost only about the 
children with whom 
he worked 


1 




Much about the children 
with whom he worked 


2 




little about the children 
with ^om he worked 


2 




Not at all about the 
children with ^om he 
worked 


4 




e. His description of the 
children 






absolutely positive 


1 




mainly positive 


2 " 




neutral 


2._ 





mainly negative 
absolutaisr negative 



Additional remarks: 
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Rating Form of the Inter*’/! ew 
Type B 



Interviewee 



Name of Rater 



Place 



Date 



Form of Interview 12 3 4 

No. of question 

1234567 

1. Answer — expansive with 

respect to question 1 

To the question but also 

going beyond 2 ^ 

Restricted with respect 
to question 3 

Evades the question 4 



8 9 



2. Much hesitation ^ 

Careful 2 

Unhesitating 2. 

Exaggerated confidence 4 



3 . Expresses himself with 

difficulty 1. 

Expresses himself with 

some difficulty ^ 

Fluent (without any 
difficulty) 2. 

Exaggerated fluency 4 

4 . Expresses himself very 

unclearlj’- 1 

Quite unclearly 2 

Clearly 2 

Very clearly 4 



5 . Special synptoms of 
nervousness (like 
biting the nails, etc.) 1 



Usual symptoms of 
nervousness but to a 
conspicuous extent 2 

Usual sytnptoms of 
nervousness to a 
limited extent 2 

Without any symptoms of 
ner*vousneSs 

Expressionless 2 



Description of the 
^miptoms of nervousness: 



Additional remarks: 



THEORY OF DATA 



We can conveniently think of research as having five phases. The 
first phase is that of selecting a universe of potential observations — or, 
in our earlier terms, a domain of investigation. The criterion for choosing 
a domain tor investigation (vihich, of course, is different from choosing 
facets to specify the domain) is simply that of personal preference. 

The second phase is that of recording observations made within the 
selected domain. In this phase one must choose what observations to make. 

It is at this stage that facet design is especially helpful . One doe s not 
wish to find that he has omitted to make observations on a vital part of 
the domain, nor does he wish to find that he has wasted his time being re- 
dundant . 

The third phase is to convert one's observations into data. If 
this sounds like an odd statement, lot me give a brief example. Suppose 
that we have recorded the fact that John was not admitted to Rutgers. I 
can think of at least four conclusions which could be drawn from this ob- 
servation: (l) John is not smart enough to get into Rutgers, (2) John is 

too far off to the side of genius for Rutgers' screening test to recognize, 
(3) Rutgers' standards are too high for John, (4) Rutgers is too stupid to 
recognize a good man when they see one. Two of these interpretations tell 
us opposite things about John, and two of them tell us opposite things 
about uiitgers. Into which datum should we convert our observation? If 
the infoimaation that John was not admitted to Rutgers appeared on a ques- 
tionnaire containing other items about John, it would no doubt be used to 
assess tha characteristics of John. If it appeared on a questionnaire sent 
out to people who had been and had nob been admitted to Rutgers, it would 
probably be used to come to some conclusion about Rutgers. In brief, the 
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kind or data we make out of our observations is not a foregone conclusion^ 
but rather a choice which the investigator must make — a choice, in fact, 
which he does make, consciously or unconsciously. 

The fourth phase is to take the data and classify the individuals 
or objects to idiich the data refer in some way Tidiich will simplify thinking 
about the data. This process is often called scaling . One technique often 
used is simply to add up the number of data which are interpreted as pointing 
in a certain direction. Very often this phase is followed by a fifth 
phase which consists of seeking relations which might exist between one 
classification of data or another classification; in other words, seeking 
relations among variables. 

The theory of data deals with the stages of making data out of 
observations and of making variables out of data. Although over the years 
a great amount of thought has gone into various aspects of these phases of 
research, it is only recently that a schems has been proposed (by Coombs) 
which promises to encompass all the facets encountered in these problems. 

If this seems too strong a statement, let me say at least that Coombs has 
proposed some facets which seem to be encountered whenever we convert ob- 
servations into data and vihenever we convert data into variables, and that 
these facets enable us to compare one with another the great variety of 
methods now in existence.; Furthermore, Coombs' theory of data systematizes 
the field in such a way that we can even use it to teU. ourselves how to 
invent new methods of collecting data and how to choose appropriate methods 
for analyzing data gathered by old or new methods . I want to carry my 
explanation in this paper far enou^ so that you can see how the theory can 
enable you to invent seme new methods of data collection. 

Some people feel that the method of data collection is of little 
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importance. Jjet me give a small example of a way in which one can come to 
quite different conclusions from identically the same actual situation be- 
cause of different methods of data collection. Let us suppose that there 
are three candidates among whom one is to be elected. Suppose that ten 
voters prefer the candidates in the order ABC as shown in the tabulation be- 
low, ten others prefer them in the order BCA and a third ten prefer them in 

Voter Preference 
type order 



1 A to B to C 

2 B to C to A 

3 C to A to B 

the order CAB. The chairman takes a vote and finds that the candidates have 
fallen into a three-way tie. Thereupon the chairman adopts a run-off method 
and asks the voters first to choose between candidates A and B. As we see 
from the tabulation, the first and the third groups of ten voters prefer A 
to B, while only the second group of ten voters prefers B to A. B is there- 
fore eliminated, and you can see that C would get twenty votes to A ' s ten 
in the run-off between A and C. Thus, C wins. 

But suppose that the chairman had first decided to pit B against C. 
The first and second groups of ten would have voted for B over C while only 
the third group would have voted for C over B. Thus, C would have been eli- 
minated. In the run-off between A and B, A would have won! Similarly, B 

would win the election if the chairman were to begin with a contest between 
A and C. In brief, the winner of the election in this case depends entirely 
upon how the cliairman decides to collect the data concerning the voter pre- 
ferences. (It should be noted also that this example does not depend upon 
having equal groups of voters. The same conclusions can be reached if any 
two groups of voters outnumber those in the third group.) 
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The theory of data begins with the Interpretation that any datum can 
be represented by a relation on a pair of points (or on a pair of pairs of 
points). For exEimplej, if we are interested in the characteristics of John we 
can call John a point representing an "individual” and Rutgers a point repre^ 
senting an object or a "stimulus." Wo can then represent Uie observation 
that John was not admitted to Rutgers by saying the relation between the two 
points is that the "individual" point is on the "negative" side of the "sti- 
mulus" point. If we are interested in evaluating Rutgers ^ we would call 
Rutgers the individu/il point and John the stimulus point. 

The theory of data is built principally on only three facets, each 
one of which contains only two elements. The first facet asks whether the 
datum is to be represented as a relation on just one pair of points or whether 
it is to be represented by a relation on two pairs of points; that is, by a 
relation on two distances. The second facet asks idiether the datum is to be 
represented by points from one set or from two sets; that is, points from a 
set representing (a) only individuals or only stimuli, or (b) points from 
both sets. The third facet asks idiether the relation is an order relation or 
a proximity relation. Coombs lays out these facets in a fom similar to that 
shown in the diagram headed "The Data Quadrants." The elements of the first 
two facets specify the "quadrants" and the elements of the third facet cor- 
respond to the "a" and "b" within each quadrant. Some examples of data are 
given within each of the quadrants of the diagram. 

It turns out that most methodologists have preferred to collect data 
by methods customarily interpreted as belonging in Quadrant II . Methods 
typically used to collect data for this quadrant are the mental test methods, 
attitude questionnaires, "pick one," rating scale methods,* magnitude esti- 
mation, and others. Quadrant III has received increasing attention in recent 
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years, while Quadrant IV has had so far the least attention of the four. In 
this paper I shall limit myself to telling you about Quadrant I. 

Quadrant I provides the model for data which are interpreted as mean- 
ing that an individual prefers object A to object B. Of course, we need not 
limit ourselves to asking an individual about only two objects at a time. For 
example, we can ask John, "What is your rank-order preference among Rutgers, 
Harvard, Princeton, and Yale?" Sometimes in a conversation we ask a person 
about his preferences among three or four objects and then, if we feel we do 
not have enou^ information, we ask him about a different assortment of three 
or four objects. We mi^t go on to ask John, "Well, how do you feel about 
Rutgers, Princeton, Michigan, and Oregon?" In other words, we can present 
the individual with different assortments of objects among which we want him 
to tell us his preferences. 

We need not ask the individual for a rank-oixier. Simetimes we mi^t 
ask a young man, "To which colleges out of this list did you apply?" If 
he has picked three out of the list we can interpret this behavior as telling 
us that those three colleges were closer to his ideal than the remainder on 
his list. These examples suggest quite a few different ways of collecting 
data. First, we can be concerned with n total objects. Second, we can pre- 
sent 2 of these n objects to the individual at a time. Third, we can ask him 
to pick k out of the £ presented to him as most preferred, or ws can ask him 
■to oi*der k out of the £ presented to him. These facets within Quadrant I 
enable us to lay out an entire grid of methods of collecting data. This grid 
is called by Coombs the "searchingness structure" and is displayed on the next 
page. This chart as shown here has no certain number of rows or columns be- 
cause n is xinspecified. The quantity n is, as I said before, the total number 
of objects or stimuli about idiich one wishes to get information from his subjects. 
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THE SEARCHIMSNESS STRUCTURE 
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The numbers of stimuli ^dch can be presented at a time are arrayed across the 
top of the chart and the numbers of those stimuli ^ich the investigator asks 
the subject pick or order are arrayed down the left side of the page. Every 
cell in the diagram represents a particular way one might go about getting 
preference information from subjects. 

Of all these possible methods, only four are well known. The cell 
near the middle left side of the diagram labeled "1" is one version of the 
method of single stimuli j one presents all the stimuli in vdiich one is inter- 
ested and asks the subject to "pick one." At the top of that column, the cell 
labeled "2" represents the method of rank order; that is, "order n-1 of n." 
(This is equivalent, of course, to "order h of n." At the ri^t side of the 
chart, the cell labeled "3" represents the mbthod of pair comparisons; that 
is, "pick 1 of 2," which is equivalent to "order 1 of 2c" The cell labeled 
"4" is the method of triads, much less used than the other three; that is, 
the n stimuli are presented to the subject three at a time and he is asked 
to order two of the three, which, of course, is equivalent to ordering all 
three in each presentation. 

Beginning with only the two facets iihich determine the searchingness 
structure as displayed here, it is clear that a myriad of new ways have been 
laid out for collecting data. Furthermore, a little thought will show there 
are other facets which mi^t be added vdiich would generate even moi*e varieties 
of ways of collecting data, very few of which have as yet been explored in 
empirical work. In recent years some woik has been done with a few of the 
"new" cells in the chart and it has been amply demonstrated that some of the 
new methods are indeed considerably more "searching" than the traditional 
methods . 

The eearchingness structure, by the way, demonstrates again the power 



of tha facet idea. Psychologiets went along for decades working with the 
methods of single stimuli, rank order, and pair ccmparisons, and with occasional 
unsystematic forays into the method of triads; it was not until Coombs looked 
hard at the ways in which the conditions of data collection could vary that 
the simple chart of the search ingness structure was laid out and it immedi- 
ately became evident that a great array of methods of data collection lay as 
yet unexplored. 

The searchingness structure itself is only one example of the kind 
of idea which is generated by the basic definition of data into ei^t simple 
types. Another Important idea is the assessment of consistency and transitivity 
by means of some of the methods laid out in the searchingness structure . Still 
another is the "unfolding technique," ^ich is crucial in assessing the rela- 
tions subjects preceive among objects after you obtain from them their pre- 
ference ordess. 

But theia is not space to go on talking about the theory of data 
here. Instead, before coming to an end, let me illustrate only one kind of 
thing which can be done with the concepts genex'ated from Quadrant I of the 
data types. Reproduced below is a passage idiioh will appear in a forthcom- 
ing book by Newcomb et al. The study described made use of the method of 
triads and the unfolding technique. The unfolding technique was elaborated 
to produce an index of cognitive similarity called "collinearity." 
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Why does a speaker "get aci^oss" better to one listener 
than to another? There cam be a number of reasons, of course, 
but surely one important i'actor in tramsmitting a meaning from 
one person to another is the extent to vdiiich the two persons pay 
attention to the same aspects of the thing they are talking about . 
A person pays attention to cej?tain qualities or aspects of 
environmental objects in preference to other qualities or aspects 
has a distinctive "cognitive structure." If we find two people 
paying attention to the same features or thread? in their conver- 
sation, we would say they are approaching the conversation with 
similar cognitive structures and we would predict that the trans- 
mission of meaning from the one person to the other would be more 
complete or quicker than in the case of two persons attempting to 
converse with dissimilar cognitive structures. This hypothesis 
was tested by Runkel (1956) in a study examining the transmission 
of meaning from instructors to college students. 

The reasoning behind this experiment was simple. If 
students are to get good grades, they must learn idiat kind of 
statements about the subject matter the teacher prefers; for 
exai^ple, they must be able to pick out pretty accurately the 
statements on a multiple choice test which the teacher himself 
would mark "right . " The student need not agree with the state- 
ments which the instructor favors; he need only be able to se^- 
lect or recognize such statements. Presumably the instructor 
judges statements according to certain characteristics or "di- 
mensions" and so also does the student. The hypothesis, then, 
is that meaning will get more quickly or more fully from teacher 
to student if the two are paying attention to the same character- 
istics or dimensions of the statements than will be the case when 
they are paying attention to different dimensions — and this will 
be true regardless of iidiether teacher and student agree on vdilch 
statements are beii^t. 

An analogy may make this more clear. Figure 2 below shows 
a room full of people. Some are young females (YF); some are old 
females (OF); some young males (IM); and some old males (OM). 
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Suppose that you walk into this room looking for someone of your 
own age with yAiom to talkj; if so, you would pay attention primarily 
to the east and west halves of the rocM because the old people are 
mostly in the east half and the young people in the west half. 

That is, you would pay attention primarily to the "east-west di- 
mension." On the other hand, if your >hief interest was to find 
someone of the opposite sex (vdiichever your own), you would pay 
attention primarily to the north and south halves of the room — 
the "north-south dimensionv" 

Suppose you were in one comer (any comer) of the room 
with a friend and he said, " I think I will look for some people 
more interesting than this bunch." In order to predi t the dir- 
ection in vdiich he would stroll away, you would have to know the 
dimension by which he judged interestingness. But you mi^t 
want to be helpful to your friend and you might suggest to him 
one or two persons with ^om he would like to converse. If you 
took it for granted that he wuld move in the same direction 
you would move and if you were right in this assimiption, then 
he would find your suggestions to be good ones. If you were 
wrong about your assumption — if your "cognitive structure" 
about the interestingness of the people in the room was differ- 
ent from his — then he would not enjoy your suggestions. 

You and your fhiend would not need to prefer the same 
kind of conversationalists in order to comprehend the preferences 
of each other. If you were twenty years old and he were fifty, 
you mifi^t suggest to him that he strike up a conversation with 
some of the people in the east half of the roomj even thou^ you 
mi^t yourself prefer to talk with young people, you might easily 
accept the idea that he would prefer to talk with, older people. 

But you would both be using the east-west dimension as a basis 
for understanding the preferences of the other person. 

We can just as well think of Figure 2 as being a field 
of statements about the realm of psychology as covered by a 
course in college. In getting ready for his experiment, Runkel 
constructed statements about the realm of psychology which could 
be imarf.ned to be scattered over a field similar to that of Fig- 
ure 2 (but no doubt having more than two dimensions). From the 
preferences among five statements expressed by students and in- 
structors, the rank order of each person's preferences was work- 
ed out. The analogy here would be walking through the room, mak- 
ing judgments about idiether one person or another was the more 
interesting conversationalist. You could very well end by rank 
ordering quite a number of the persons from the most to least 
interesting. 

By using a geometric model (Coombs, 1964) it was possible 
to estimate Aether, on the one hand, a teacher and his student 
were probably judging the statements about psychology on the same 
dimension or, on the other hand^ idiether they were using different 
dimensions. If the instructor and student were basing their pref- 
erences among the statements on the same dimension, this was a 
rou^ index of their "cognitive similarity" about psychology; if 
they were using different dimensions, this indicated cognitive 
dissimilarity. 
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Qualities of statements about psychology are not at all 
as easy to perceive as age and sex of human beings in a room. 

It is not easy to discern the dimensions of preference of another 
person conceniing complicated matters such as opinions about psy- 
chological topics. The hypothesis in the experiment was that 
students >dio showed cognitive similarities to their instructors 
in regard to the statements would be better able to catch on to 
the preferences of their instructors about such statements^ while 
students with cognitive structures dissimilar to those of their 
instructors would have a harder time anticipating the statements 
their instructors would prefer. Consequently, the prediction was 
that the "cognitiveTysdmilar*' students would obtain higher grades 
on quizzes written and graded by their instructors than would the 
" cognitivel/ <fis similar " student s . 

The results computed for those subjects who were stable 
in their opinions (both in respect to their instructors and over 
the time period of a semester) supported the prediction and are 
summarized in the table. Because grading is different in level 
and range from class to class, the quiz scores were converted to 
"standard" scores within each class; that is, they were recomputed 
so that the standard deviation within each class became the unit 
for scoring. These standards scores are the "z-scores" mentioned 
in Table 3 below. The mean z-score for students with cognitive 
structures similar to those of their instructors was significantly 
hi^er (P <.05) than the mean for students with cognitive struc- 
tures dissimilar from their instructors. 

Table 2.* Difference between z-scores on quizzes for student?; whose 
cognitive structure about psychology was (a) similar to that of 
their instructor and for those whose cognitive structure was 
dissimilar to that of their instructor. 





(a) 

Similar 


(b) 

Dissimilar 


Top z-score 


2.77 


1.74 


Mean z-score 


0.60 


-0.25 


Lowest z-score 


- 1.16 


- 2.56 


Standard deviation 


1.38 


1.17 


Number of students 


17 


19 



A number of comnon causes of differences between quiz 
scores were examined in this experiment by looking further into 
the data. No evidence was found to cast doubt upon the hypo- 
thesis concerning cognitive similarity. 
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This paper is already too long and I shall end my list of fresh 
ideas in research methodology with only the two I have described. There 
are certainly many other important ideas coming to notice these days 
which are esctremeiy helpful to the researcher. One should certainly men- 
tion the scheme for thinking about control group£< and experimental design 
idiich is put forward by Donald T. Campbell and Julian C. Stanley as a 
chapter of the Handbook for Research on Teaching edited by N. L. Gage. 

One should also mention the new explorations into Bayesian statistics. 
Another important trend is the renewed and more sophisticated attention 
being given to the effect of the observer or experimenter on what is be- 
ing observed. But one must call a halt. 

If you have time to try another exercise you might try this one: 

EXERCISE 2. Describe one or two facets which mi^t be added to 
the searchingness structure to extend the catalogue of methods 
of data collection. 

Some selected references follow: 
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EFFECTS OF OBSERVERS 



McGud.gan, F. J. The experimenter: a neglected stimulus object. Psychol . 

Bull .. 1963, 60, 421-428. 

Ome, Martin T. On the social psychology of the psychological experiment: 
with particular reference to demand characteristics and their 
implications. Amer . Psychol . . 1962, 17, 776-783. 

Rosenthal, Robert; Persinger, Gordon W.; Kline, Linda Vikan; & Mulry, Ray C. 

The role of the research assistant in the mediation of experimenter 
bias. J. Pers., 1963, 31, 313-335. 

Sherif , Muzafer & Sherif , Carolyn W. Reference groups . New York: Harper 

& Row, 1964. 

NOTE: The facet designs for (a) job classification and (b) inter- 
viewing applicants for training in coimseling are taken from informal mate- 
rials distributed by Guttman in a seminar held some years ago at the University 
of Illinois; thqy should not be cited or quoted as representing current prac- 
tice in Israel. 
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